Systems and methods for multi-label segmentation of cardiac computed tomography and angiography images using deep neural networks

ABSTRACT

Methods and systems are provided for detecting coronary lesions in 3D cardiac computed tomography and angiography (CCTA) images using deep neural networks. In an exemplary embodiment, a method for detecting coronary lesions in 3D CCTA images comprises, acquiring a 3D CCTA image of a coronary tree, mapping the 3D CCTA image to a multi-label segmentation map with a trained deep neural network, generating a plurality of 1D parametric curves for a branch of the coronary tree using the multi-label segmentation map, determining a location of a lesion in the branch of the coronary tree using the plurality of 1D parametric curves, and determining a severity score for the lesion based on the plurality of 1D parametric curves.

CROSS REFERENCE TO RELATED APPLICATIONS

The present application claims priority to U.S. non-provisionalapplication Ser. No. 16/653,906, entitled “SYSTEMS AND METHODS FORMULTI-LABEL SEGMENTATION OF CARDIAC COMPUTED TOMOGRAPHY AND ANGIOGRAPHYIMAGES USING DEEP NEURAL NETWORKS”, filed on Oct. 15, 2019. The entirecontents of the above-listed application are hereby incorporated byreference for all purposes.

TECHNICAL FIELD

Embodiments of the subject matter disclosed herein relate to analyzingcardiac computed tomography and angiography (CCTA) images, and moreparticularly, to systems and methods for multi-label segmentation ofCCTA images using deep neural networks.

BACKGROUND

Today, the assessment of coronary lesions is done manually byclinicians. The review of cardiac computed tomography and angiography(CCTA) images to determine the condition of a the coronary arteries is atedious task because the coronary arteries are nonplanar,three-dimensional (3D) structures, and the clinician may need to rely onadvanced visualization tools to view the 3D structures in multipleperspective views, such as along a longitudinal axis of a coronaryartery, along a cross section of a coronary artery etc., to assess thepresence/severity of a coronary lesion. As a consequence, theidentification of coronary lesions and the assessment of thetype/severity of coronary lesions is labor intensive and time consuming.Further detection and assessment of coronary lesions in the mannerdescribed above may also produce inconsistent diagnoses, especially whenthe assessment on CCTA images is compared to other imaging modalitiessuch as intracardiac ultrasound, optical coherence tomography andangiography (OCTA), etc. Thus, exploring techniques for rapidly andaccurately detecting coronary lesions using CCTA images, and determininga type/severity of the detected lesions, is generally desired.

SUMMARY

The present disclosure at least partially addresses the issues describedabove. In one embodiment, a method for detecting coronary lesions in 3DCCTA images comprises, acquiring a 3D CCTA image of a coronary treemapping the 3D CCTA image to a multi-label segmentation map with atrained deep neural network, generating a plurality of one-dimensional(1D) parametric curves for a branch of the coronary tree using themulti-label segmentation map, determining a location of a lesion in thebranch of the coronary tree using the plurality of 1D parametric curves,and determining a severity score for the lesion based on the pluralityof 1D parametric curves. In this way, automatic and robust detection ofcoronary lesions may be enabled.

The above advantages and other advantages, and features of the presentdescription will be readily apparent from the following DetailedDescription when taken alone or in connection with the accompanyingdrawings. It should be understood that the summary above is provided tointroduce in simplified form a selection of concepts that are furtherdescribed in the detailed description. It is not meant to identify keyor essential features of the claimed subject matter, the scope of whichis defined uniquely by the claims that follow the detailed description.Furthermore, the claimed subject matter is not limited toimplementations that solve any disadvantages noted above or in any partof this disclosure.

BRIEF DESCRIPTION OF THE DRAWINGS

The patent or application file contains at least one drawing executed incolor. Copies of this patent or patent application publication withcolor drawing(s) will be provided by the Office upon request and paymentof the necessary fee.

Various aspects of this disclosure may be better understood upon readingthe following detailed description and upon reference to the drawings inwhich:

FIG. 1 is an illustration of an exemplary embodiment of a cardiaccomputed tomography and angiography (CCTA) imaging system;

FIG. 2 shows a block diagram of an exemplary embodiment of an imagingsystem;

FIG. 3 shows an exemplary embodiment of a CCTA image processing system.

FIG. 4 is a schematic diagram illustrating an exemplary architecture ofa deep neural network which can be used in the system of FIG. 3,according to an exemplary embodiment;

FIG. 5 is a flow chart illustrating a method for detecting coronarylesions in 3D CCTA images using deep neural networks, according to anexemplary embodiment;

FIG. 6 is a flow chart illustrating a method for training a deep neuralnetwork to determine a multi-label segmentation map from a 3D CCTAimage, according to an exemplary embodiment;

FIG. 7 is a prophetic example of 1D parametric curves which may begenerated from a multi-label segmentation map for a branch of a coronarytree; and

FIG. 8 shows examples of CCTA images with corresponding multi-labelsegmentation maps, according to an exemplary embodiment of the currentdisclosure.

The drawings illustrate specific aspects of the described systems andmethods for detecting coronary lesions using 3D CCTA images and deepneural networks. Together with the following description, the drawingsdemonstrate and explain the structures, methods, and principlesdescribed herein. In the drawings, the size of components may beexaggerated or otherwise modified for clarity. Well-known structures,materials, or operations are not shown or described in detail to avoidobscuring aspects of the described components, systems and methods.

DETAILED DESCRIPTION

The following description relates to various embodiments for determining3D multi-label segmentation maps from 3D CCTA images using deep neuralnetworks. The disclosure further relates to automatically detectingcoronary lesions using the 3D multi-label segmentation maps. In oneembodiment, 3D CCTA images acquired by CCTA imaging system 100, shown inFIG. 1, or imaging system 200, shown in FIG. 2, may be analyzed for thepresence of coronary lesions by CCTA image processing system 302, shownin FIG. 3. CCTA image processing system 302 may execute one or moresteps of method 500, shown in FIG. 5, which includes mapping theacquired 3D CCTA image to a multi-label segmentation map usingconvolutional neural network 400, shown in FIG. 4, and using themulti-label segmentation map to determine a plurality of 1D parametriccurves for each branch of the coronary tree. An example of 1D parametriccurves which may be generated using multi-label segmentation maps areshown in FIG. 7, while examples of CCTA images and correspondingmulti-label segmentation maps are shown in FIG. 8. The plurality of 1Dparametric curves may be used to detect one or more coronary lesions,and to assess a severity of the one or more coronary lesions. Theconvolutional neural network 400 used to map the 3D CCTA images tomulti-label segmentation maps may be trained using method 600, shown inFIG. 6, wherein training data pairs comprising 3D CCTA images andcorresponding ground truth multi-label segmentation maps are used toadjust one or more parameters of the deep neural network.

As used herein, 3D CCTA images refer to three-dimensional CCTA imagescomprising a measured volume. 3D CCTA images comprise imaging data forthree distinct spatial dimensions. 3D CCTA images may comprise aplurality of voxels, or units of volume, which may be contrasted withpixels of two-dimensional images, wherein pixels comprise units of area.Similarly, 3D multi-label segmentation maps comprise label data for oneor more voxels of the 3D CCTA images, and therefore may comprise labelsfor voxels arranged in a three-dimensional space.

As used herein, 1D parametric curves refer to parameters or valuesmeasured along a single dimension, such as length, or angle. A 1Dparametric curve shows a parameter as a function of a single spatialdimension, and may be illustrated graphically as a series of values of aparameter at each of a plurality of set intervals along the spatialdimension.

FIG. 1 illustrates an exemplary CCTA imaging system 100 configured toallow fast and iterative image reconstruction of the coronary arteriesof a heart. Particularly, the CCTA imaging system 100 is configured toimage a heart of subject 112. In one embodiment, the CCTA imaging system100 includes a gantry 102, which in turn, may include at least one x-rayradiation source 104 configured to project a beam of x-ray radiation 106for use in imaging the subject 112. Specifically, the x-ray radiationsource 104 is configured to project the x-rays 106 towards a detectorarray 108 positioned on the opposite side of the gantry 102. AlthoughFIG. 1 depicts only a single x-ray radiation source 104, in certainembodiments, multiple x-ray radiation sources and detectors may beemployed to project a plurality of x-rays 106 for acquiring projectiondata corresponding to the patient at different energy levels. In someembodiments, the x-ray radiation source 104 may enable dual-energygemstone spectral imaging (GSI) by rapid kVp switching. In someembodiments, the x-ray detector employed is a photon-counting detectorwhich is capable of differentiating x-ray photons of different energies.In other embodiments, two sets of x-ray tube-detector are used togenerate dual-energy projections, with one set at low-kVp and the otherat high-kVp. It should thus be appreciated that the methods describedherein may be implemented with single energy acquisition techniques aswell as dual energy acquisition techniques.

In certain embodiments, the CCTA imaging system 100 further includes animage reconstruction unit 110, configured to reconstruct images of atarget volume of the subject 112 using an iterative or analytic imagereconstruction method. For example, the image reconstruction unit 110may use an analytic image reconstruction approach such as filteredbackprojection (FBP) to reconstruct images of a target volume of thepatient. As another example, the image reconstruction unit 110 may usean iterative image reconstruction approach such as advanced statisticaliterative reconstruction (ASIR), conjugate gradient (CG), maximumlikelihood expectation maximization (MLEM), model-based iterativereconstruction (MBIR), and so on, to reconstruct images of a targetvolume of the subject 112. In some embodiments, the image processor unit110 may use both an analytic image reconstruction approach such as FBPin addition to an iterative image reconstruction approach.

In some known CCTA imaging system configurations, a radiation sourceprojects a cone-shaped beam which is collimated to lie within an X-Y-Zplane of a Cartesian coordinate system and generally referred to as an“imaging plane.” The radiation beam passes through an object beingimaged, such as the patient or subject 112. The beam, after beingattenuated by the object, impinges upon an array of radiation detectors.The intensity of the attenuated radiation beam received at the detectorarray is dependent upon the attenuation of a radiation beam by theobject. Each detector element of the array produces a separateelectrical signal that is a measurement of the beam attenuation at thedetector location. The attenuation measurements from all the detectorsare acquired separately to produce a transmission profile.

In some CCTA imaging systems, the radiation source and the detectorarray are rotated with a gantry within the imaging plane and around theobject to be imaged such that an angle at which the radiation beamintersects the object constantly changes. A group of radiationattenuation measurements, i.e., projection data, from the detector arrayat one gantry angle is referred to as a “view.” A “scan” of the objectincludes a set of views made at different gantry angles, or view angles,during one revolution of the radiation source and detector.

The projection data is processed to reconstruct an image thatcorresponds to a two-dimensional slice taken through the object. Onemethod for reconstructing an image from a set of projection data isreferred to in the art as the filtered backprojection technique.Transmission and emission tomography reconstruction techniques alsoinclude statistical iterative methods such as maximum likelihoodexpectation maximization (MLEM) and ordered-subsetsexpectation-reconstruction techniques as well as iterativereconstruction techniques. This process converts the attenuationmeasurements from a scan into integers called “CT numbers” or“Hounsfield units,” which are used to control the brightness of acorresponding pixel on a display device.

To reduce the total scan time, a “helical” scan may be performed. Toperform a “helical” scan, the patient is moved while the data for theprescribed number of slices is acquired. Such a system generates asingle helix from a cone beam helical scan. The helix mapped out by thecone beam yields projection data from which images in each prescribedslice may be reconstructed.

As used herein, the phrase “reconstructing an image” is not intended toexclude embodiments of the present invention in which data representingan image is generated but a viewable image is not. Therefore, as usedherein the term “image” broadly refers to both viewable images and datarepresenting a viewable image. However, many embodiments generate (orare configured to generate) at least one viewable image.

FIG. 2 illustrates an exemplary imaging system 200 similar to the CCTAimaging system 100 of FIG. 1. In one embodiment, the imaging system 200includes the detector array 108 (see FIG. 1). The detector array 108further includes a plurality of detector elements 202 that togethersense the x-ray beams 106 (see FIG. 1) that pass through a subject 204such as a patient to acquire corresponding projection data. Accordingly,in one embodiment, the detector array 108 is fabricated in a multi-sliceconfiguration including the plurality of rows of cells or detectorelements 202. In such a configuration, one or more additional rows ofthe detector elements 202 are arranged in a parallel configuration foracquiring the projection data.

In certain embodiments, the imaging system 200 is configured to traversedifferent angular positions around the subject 204 for acquiring desiredprojection data. Accordingly, the gantry 102 and the components mountedthereon may be configured to rotate about a center of rotation 206 foracquiring the projection data, for example, at different energy levels.Alternatively, in embodiments where a projection angle relative to thesubject 204 varies as a function of time, the mounted components may beconfigured to move along a general curve rather than along a segment ofa circle.

As the x-ray source 104 and the detector array 108 rotate, the detectorarray 108 collects data of the attenuated x-ray beams. The datacollected by the detector array 108 undergoes pre-processing andcalibration to condition the data to represent the line integrals of theattenuation coefficients of the scanned subject 204. The processed dataare commonly called projections.

In dual or multi-energy imaging, two or more sets of projection data aretypically obtained for the imaged object at different tube kilovoltage(kVp) levels, which change the maximum and spectrum of energy of theincident photons comprising the emitted x-ray beams or, alternatively,at a single tube kilovoltage (kVp) level or spectrum with an energyresolving detector of the detector array 108.

The acquired sets of projection data may be used for basis materialdecomposition (BMD). During BMD, the measured projections are convertedto a set of material-density projections. The material-densityprojections may be reconstructed to form a pair or a set ofmaterial-density map or image of each respective basis material, such asbone, soft tissue, and/or contrast agent maps. The density maps orimages may be, in turn, associated to form a volume rendering of thebasis material, for example, bone, soft tissue, and/or contrast agent,in the imaged volume.

In CCTA imaging, a radio contrast agent is administered to a patientprior to imaging, the radiocontrast agent enables differentiationbetween vessel walls and lumen to be made, as the x-ray beams passingthrough radiocontrast agent are differently attenuated than thosepassing through other materials, such as plaques or vessel walls.

Once reconstructed, the basis material image produced by the imagingsystem 200 reveals internal features of the subject 204, expressed inthe densities of the two basis materials. The density image may bedisplayed to show these features. In traditional approaches to diagnosismedical conditions, such as disease states, and more generally ofmedical events, a radiologist or physician would consider a hard copy ordisplay of the density image to discern characteristic features ofinterest. Such features might include lesions, sizes and shapes ofparticular anatomies or organs, and other features that would bediscernable in the image based upon the skill and knowledge of theindividual practitioner.

In one embodiment, the imaging system 200 includes a control mechanism208 to control movement of the components such as rotation of the gantry102 and the operation of the x-ray radiation source 104. In certainembodiments, the control mechanism 208 further includes an x-raycontroller 210 configured to provide power and timing signals to theradiation source 104. Additionally, the control mechanism 208 includes agantry motor controller 212 configured to control a rotational speedand/or position of the gantry 102 based on imaging requirements.

In certain embodiments, the control mechanism 208 further includes adata acquisition system (DAS) 214 configured to sample analog datareceived from the detector elements 202 and convert the analog data todigital signals for subsequent processing. The data sampled anddigitized by the DAS 214 is transmitted to a computer or computingdevice 216. In one example, the computing device 216 stores the data ina storage device 218. The storage device 218, for example, may include ahard disk drive, a floppy disk drive, a compact disk-read/write (CD-R/W)drive, a Digital Versatile Disc (DVD) drive, a flash drive, and/or asolid-state storage drive.

Additionally, the computing device 216 provides commands and parametersto one or more of the DAS 214, the x-ray controller 210, and the gantrymotor controller 212 for controlling system operations such as dataacquisition and/or processing. In certain embodiments, the computingdevice 216 controls system operations based on operator input. Thecomputing device 216 receives the operator input, for example, includingcommands and/or scanning parameters via an operator console 220operatively coupled to the computing device 216. The operator console220 may include a keyboard (not shown) or a touchscreen to allow theoperator to specify the commands and/or scanning parameters.

Although FIG. 2 illustrates only one operator console 220, more than oneoperator console may be coupled to the imaging system 200, for example,for inputting or outputting system parameters, requesting examinations,and/or viewing images. Further, in certain embodiments, the imagingsystem 200 may be coupled to multiple display devices, printers,workstations, and/or similar devices located either locally or remotely,for example, within an institution or hospital, or in an entirelydifferent location via one or more configurable wired and/or wirelessnetworks such as the Internet and/or virtual private networks.

The computing device 216 uses the operator-supplied and/orsystem-defined commands and parameters to operate a table motorcontroller 226, which in turn, may control a table 228 which maycomprise a motorized table. Particularly, the table motor controller 226moves the table 228 for appropriately positioning the subject 204 in thegantry 102 for acquiring projection data corresponding to the targetvolume of the subject 204.

As previously noted, the DAS 214 samples and digitizes the projectiondata acquired by the detector elements 202. Subsequently, an imagereconstructor 230 uses the sampled and digitized x-ray data to performhigh-speed reconstruction. Although FIG. 2 illustrates the imagereconstructor 230 as a separate entity, in certain embodiments, theimage reconstructor 230 may form part of the computing device 216.Alternatively, the image reconstructor 230 may be absent from theimaging system 200 and instead the computing device 216 may perform oneor more functions of the image reconstructor 230. Moreover, the imagereconstructor 230 may be located locally or remotely, and may beoperatively connected to the imaging system 200 using a wired orwireless network. Particularly, one exemplary embodiment may usecomputing resources in a “cloud” network cluster for the imagereconstructor 230.

In one embodiment, the image reconstructor 230 stores the imagesreconstructed in the storage device 218. Alternatively, the imagereconstructor 230 transmits the reconstructed images to the computingdevice 216 for generating useful patient information for diagnosis andevaluation. In certain embodiments, the computing device 216 transmitsthe reconstructed images and/or the patient information to a display 232communicatively coupled to the computing device 216 and/or the imagereconstructor 230.

The various methods and processes described further herein may be storedas executable instructions in non-transitory memory on a computingdevice in imaging system 200.

In one embodiment, the display 232 allows the operator to evaluate theimaged anatomy. The display 232 may also allow the operator to select avolume of interest (VOI) and/or request patient information, forexample, via a graphical user interface (GUI) for a subsequent scan orprocessing.

Referring to FIG. 3, an exemplary embodiment of a CCTA image processingsystem 302 is shown. In some embodiments, CCTA image processing system302 may be communicatively coupled to one or more of CCTA imaging system100, or imaging system 200, shown in FIGS. 1 and 2, respectively. Insome embodiments CCTA image processing system 302 is incorporated intothe CCTA imaging system. In some embodiments, at least a portion of CCTAimage processing system 302 is disposed at a device (e.g., edge device,server, etc.) communicably coupled to the CCTA imaging system via wiredand/or wireless connections. In some embodiments, at least a portion ofCCTA image processing system 302 is disposed at a separate device (e.g.,a workstation) which can receive images from the CCTA imaging system orfrom a storage device which stores the images generated by the CCTAimaging system. CCTA image processing system 302 comprises processor304, configured to execute instructions stored in non-transitory memory306. CCTA image processing system 302 is communicatively coupled todisplay device 326, and user input device 324, which may enable a userto view and input/interact with data stored within CCTA image processingsystem 302, respectively.

CCTA image processing system 302 includes a processor 304 configured toexecute machine readable instructions stored in non-transitory memory306. Processor 304 may be single core or multi-core, and the programsexecuted thereon may be configured for parallel or distributedprocessing. In some embodiments, the processor 304 may optionallyinclude individual components that are distributed throughout two ormore devices, which may be remotely located and/or configured forcoordinated processing. In some embodiments, one or more aspects of theprocessor 304 may be virtualized and executed by remotely-accessiblenetworked computing devices configured in a cloud computingconfiguration.

Non-transitory memory 306 may store deep neural network module 308,training module 310, CCTA image data 312, parametric curve module 314,and lesion detection module 316. Non-transitory memory may comprisecomputer-readable media (CRM) that stores data in the presence orabsence of power. In some embodiments, the non-transitory memory 306 mayinclude components disposed at two or more devices, which may beremotely located and/or configured for coordinated processing. In someembodiments, one or more aspects of the non-transitory memory 306 mayinclude remotely-accessible networked storage devices configured in acloud computing configuration.

Deep neural network module 308 may include one or more deep neuralnetworks, comprising a plurality of weights and biases, activationfunctions, and instructions for implementing the one or more deep neuralnetworks to receive 3D CCTA images and map the 3D CCTA images to 3Dmulti-label segmentation maps. For example, deep neural network module308 may store parameters of, and instructions for implementing, a neuralnetwork, such as the convolutional neural network (CNN) 400 illustratedin FIG. 4. Deep neural network module 308 may include trained and/oruntrained neural networks and may further include various data, such asmetadata pertaining to the one or more deep neural networks stored indeep neural network module 308.

Non-transitory memory 306 may further include training module 310, whichcomprises instructions for training one or more of the deep neuralnetworks stored in deep neural network module 308. Training module 310may include one or more gradient descent algorithms, one or more lossfunctions, one or more training data selection criteria, one or morebackpropagation algorithms, etc. Training module 310 may includeinstructions that, when executed by processor 304, cause CCTA imageprocessing system 302 to conduct one or more of the steps of method 600,discussed in more detail below. In one example, training module 310includes instructions for selecting training data pairs from CCTA imagedata 312, which comprise pairs of 3D CCTA images of coronary trees, suchas may be acquired via CCTA imaging system 100, and ground truth 3Dmulti-label segmentation maps comprising ground truth labels for eachvoxel of the 3D CCTA image, wherein the labels are of one or moreanatomical classes, including lumen, media, fibrous cap, lipid core,calcium, external tissue, and undetermined. In some embodiments, theground truth multi-label segmentation maps comprise higher spatialresolution than the corresponding input CCTA images (that is, for aparticular training data pair, a number of voxels in the ground truthmulti-label segmentation map is greater than the number of voxels in thecorresponding input CCTA images), thereby enabling training of a deepneural network to produce multi-label segmentation maps of a higherspatial resolution than the native resolution of the input CCTA images(i.e., super resolution). The training data pairs selected by trainingmodule 310 may be used to conduct supervised learning of one or moredeep neural networks stored in deep neural network module 308. In someembodiments, the training module 310 is not disposed at the CCTA imageprocessing system 302.

Non-transitory memory 306 further includes CCTA image data 312. In someembodiments, CCTA image data 312 includes a plurality of 3D CCTA imagesof coronary trees, which may be indexed by one or more pieces ofmetadata pertaining to the 3D CCTA images. In some embodiments, CCTAimage data 312 may further include a plurality of training data pairs,comprising 3D CCTA images and corresponding ground truth multi-labelsegmentation maps. In some embodiments, multi-label segmentation mapsgenerated by a deep neural network may be stored in CCTA image data 312.In some embodiments, CCTA image data 312 is not disposed at the CCTAimage processing system 302.

Non-transitory memory 306 further includes parametric curve module 314,which may be configured to determine one or more parametric curves alonga centerline of a branch of a coronary tree imaged by a 3D CCTA image.In some embodiments, parametric curve module 314 comprises instructionsfor determining one or more pre-defined/pre-determined parameters ateach point along a centerline of a branch of a coronary tree, andproducing a distinct parametric curve for each pre-determined parameter.In one embodiment, parametric curve module 314 may receive a 3Dmulti-label segmentation map, comprising a plurality of probabilityscores for a plurality of anatomical classes for each voxel (orsub-voxel, in the case of super resolution) of a 3D CCTA image, and maydetermine one or more of lumen radius, plaque thickness, vessel wallthickness, etc. at each point along a centerline of a branch of thecoronary tree included within the 3D CCTA image. Parametric curve module314 may include definitions of the one or more pre-defined parameters,and one or more algorithms for calculating the parameters usingmulti-label segmentation maps. In some embodiments, parametric curvemodule 314 is not disposed at the CCTA image processing system 302.

Non-transitory memory 306 further includes lesion detection module 316,which may be configured to receive one or more 1D parametric curvesgenerated by parametric curve module 314, and determine if one or morelesions are present. In response to a determination that one or morelesions are present, lesion detection module 316 may includeinstructions for determining a location and severity score for each ofthe one or more lesions. Lesion detection module 316 may include one ormore pre-defined criteria for detection of lesions using 1D parametriccurves. In some embodiments, lesion detection module 316 may include oneor more pre-defined thresholds, wherein lesion detection module 316 maydetermine that a lesion is present in a 1D parametric curve based on acomparison between the 1D parametric curve and the pre-definedthresholds. In some embodiments, lesion detection module 316 is notdisposed at CCTA image processing system 302.

CCTA image processing system 302 may further include user input device324. User input device 324 may comprise one or more of a touchscreen, akeyboard, a mouse, a trackpad, a motion sensing camera, or other deviceconfigured to enable a user to interact with and manipulate data withinCCTA image processing system 302. In one example, user input device 324may enable a user to make a selection of a 3D CCTA image to detectlesions within, using a method, such as method 500 discussed below. Insome embodiments, a user may generate a ground truth multi-labelsegmentation map using user input device 324.

Display device 326 may include one or more display devices utilizingvirtually any type of technology. In some embodiments, display device326 may comprise a computer monitor, and may display 3D CCTA images, 3Dmulti-label segmentation maps, 1D parametric curves, locations ofdetected lesions, and severity scores of detected lesions. Displaydevice 326 may be combined with processor 304, non-transitory memory306, and/or user input device 324 in a shared enclosure, or may beperipheral display devices and may comprise a monitor, touchscreen,projector, or other display device known in the art, which may enable auser to view CCTA images produced by a CCTA imaging system, and/orinteract with various data stored in non-transitory memory 306.

It should be understood that CCTA image processing system 302 shown inFIG. 3 is for illustration, not for limitation. Another appropriateimage processing system may include more, fewer, or differentcomponents.

Turning to FIG. 4, a schematic of a convolutional neural network (CNN)400, for mapping a 3D CCTA image to a 3D multi-label segmentation map isshown, in accordance with an exemplary embodiment. The 3D multi-labelsegmentation maps output by CNN 400 may comprise a 3D array ofprobability scores, corresponding to the 3D array of voxel intensityvalues in the input 3D CCTA image. Each voxel of the input 3D CCTA imagemay be mapped to a plurality of probability scores corresponding to aplurality of pre-defined anatomical classes, wherein the probabilityscore for each anatomical class indicates a likelihood of a voxel towhich the probability score was assigned, belonging to the anatomicalclass. In one example, a first voxel of a 3D CCTA image may be mapped toa first probability score and a second probability score in a 3Dmulti-label segmentation map, wherein the first probability scoreindicates a probability that the first voxel of the 3D CCTA imagesrepresents a first anatomical class (e.g., lumen), and the secondprobability score indicates a probability that the first voxel of the 3DCCTA image represents a second anatomical class (e.g., plaque), whereinthe first and second anatomical classes are distinct.

In some embodiments, CNN 400 may comprise a greater number of outputsthan inputs, enabling each voxel of an input 3D CCTA image to be splitinto a plurality of sub-voxels, and a probability score/classificationscore, for each anatomical class, may be determined for each sub-voxel,thereby producing an output 3D multi-label segmentation map with agreater resolution than the native resolution of the input CCTA image.

CNN 400 comprises a U-net architecture, which may be divided into anencoder portion (descending portion, elements 402-430) and a decoderportion (ascending portion, elements 432-456). CNN 400 is configured toreceive 3D CCTA image data of a heart/coronary tree at input layer 402,comprising a plurality of voxel intensity values, and map the input 3DCCTA image data to a 3D multi-label segmentation map of theheart/coronary. CNN 400 transforms/maps the received 3D CCTA image databy performing a series of convolutions, activations, down samplingoperations, and up-sampling operations, and produces a 3D multi-labelsegmentation map based on output from output layer 456.

The various elements and operations of CNN 400 are labeled in legend458. As indicated by legend 458, CNN 400 includes a plurality of featuremaps (and/or copied feature maps), wherein each feature map may receiveinput from either an external file, or a previous feature map, and maytransform/map the received input to output to produce a next featuremap. Each feature map may comprise a plurality of neurons, where in someembodiments, each neuron may receive input from a subset of neurons of aprevious layer/feature map, and may compute a single output based on thereceived inputs, wherein the output may be propagated to a subset of theneurons in a next layer/feature map. A feature map may be describedusing spatial dimensions, such as length, width, depth, and hyper depth(which may correspond to features of each of voxel of the inputimage/volume), wherein the dimensions refer to the number of neuronscomprising the feature map (e.g., the number of neurons along a length,the number of neurons along a width, the number of neurons along adepth, and the number of neurons along a hyper depth of a specifiedfeature map).

In some embodiments, the neurons of the feature maps may compute anoutput by performing a dot product of received inputs using a set oflearned weights (each set of learned weights may herein be referred toas a filter), wherein each received input has a unique correspondinglearned weight, wherein the learned weight was learned during trainingusing a plurality of training data pairs.

The transformations/mappings performed by each feature map are indicatedby arrows, wherein each type of arrow corresponds to a distincttransformation, as indicated by legend 458. Rightward pointing solidblack arrows indicate 3×3×3 convolutions with stride of one, whereinoutput from a 3×3×3 grid of feature channels of an immediately precedingfeature map are mapped to a single feature channel of a current featuremap. Each 3×3×3 convolution may be followed by an activation function,wherein, in one embodiment, the activation function comprises arectified linear unit (ReLU).

Downward pointing hollow arrows indicate 2×2×2 max pooling, wherein themax value from a 2×2×2 grid of feature channels is propagated from animmediately preceding feature map to a single feature channel of acurrent feature map, thereby resulting in an 8-fold reduction in spatialresolution of the immediately preceding feature map. In some examples,this pooling occurs for each feature independently.

Upward pointing hollow arrows indicate 2×2×2 up-convolutions, whichcomprise mapping output from a single feature channel of an immediatelypreceding feature map to a 2×2×2 grid of feature channels in a currentfeature map, thereby increasing the spatial resolution of theimmediately preceding feature map 8-fold.

Rightward pointing dash-tailed arrows indicate copying and cropping of afeature map for concatenation with another, later occurring, featuremap. Cropping enables the dimensions of the copied feature map to matchthe dimensions of the feature map with which the copied feature map isto be concatenated. It will be appreciated that when the size of thefirst feature map being copied and the size of the second feature map tobe concatenated with the first feature map are equal, no cropping may beperformed.

Rightward pointing arrows with hollow elongated triangular headsindicate a 1×1×1 convolution, in which each feature channel in animmediately preceding feature map is mapped to a single feature channelof a current feature map, or in other words, wherein a 1-to-1 mapping offeature channels between an immediately preceding feature map and acurrent feature map occurs.

Rightward pointing arrows with chevron heads indicate incorporation ofGaussian noise into a received input feature map.

Rightward pointing arrows with arcuate hollow heads indicate batchnormalization operations, wherein a distribution of activations of aninput feature map are normalized.

Rightward pointing arrows with a short hollow triangular head indicatesa dropout operation, wherein random or pseudo-random dropout of inputneurons (as well as their inputs and outputs) occurs during training.

In addition to the operations indicated by the arrows within legend 458,CNN 400 includes solid filled rectangles corresponding to feature maps,wherein feature maps comprise a height (top to bottom length as shown inFIG. 4, corresponds to a y spatial dimension in an x-y plane), width(not shown in FIG. 4, assumed equal in magnitude to height, correspondsto an x spatial dimension in an x-y plane), and depth (a left-rightlength as shown in FIG. 4, corresponds to the number of features withineach feature channel). Likewise, CNN architecture 400 includes hollow(unfilled) rectangles, corresponding to copied and cropped feature maps,wherein copied feature maps comprise height (top to bottom length asshown in FIG. 4, corresponds to a y spatial dimension in an x-y plane),width (not shown in FIG. 4, assumed equal in magnitude to height,corresponds to an x spatial dimension in an x-y plane), and depth (alength from a left side to a right side as shown in FIG. 4, correspondsto the number of features within each feature channel).

Starting at 3D CCTA image 402 (herein also referred to as an inputlayer), data corresponding to a 3D CCTA image may be input and mapped toa first set of features. In one embodiment, CNN 400 comprises one neuronfor each feature, of each voxel, of the input 3D CCTA image. Forexample, for an input 3D CCTA image comprising 100×100×100 voxels, with3 color channels per voxel, input layer 402 may comprise 100×100×100×3input neurons. The 3D CCTA image may have been acquired by a CCTAimaging system, such as CCTA imaging system 100, shown in FIG. 1, or byan imaging system, such as imaging system 200, shown in FIG. 2. In someembodiments, the input data is pre-processed (e.g., normalized) beforebeing processed by CNN 400. In some embodiments, the input data is anarray of intensity values for a plurality of voxels.

Output layer 456 may comprise a plurality of output neurons, whereineach output neuron may correspond to a distinct region in space withinthe volume imaged by input 3D CCTA image. Output from each output neuronmay comprise a probability score/classification score for one or morepre-defined anatomical classes, assigned to a voxel (or sub-voxel) ofthe input 3D CCTA image. For example, the output of an output neuron mayindicate a probability that a voxel (or sub-voxel) of an input 3D CCTAimage is part of a vessel wall, a fibrous cap of a plaque, a lipid coreof a plaque, etc. In some embodiments, output layer 456 may comprise agreater spatial resolution than input layer 402. For example, if aninput layer comprises 100×100×100×F input neurons, where F is the numberof features per voxel, output layer 456 may comprise 200×200×200×O,where O is the number of anatomical classes, and where each voxel of theinput image has been divided into four sub-voxels in the output image.

The weights (and biases) of the convolutional layers in the neuralnetwork 400 are learned during training, as will be discussed in moredetail with reference to FIG. 6 below. Briefly, a loss function isdefined to reflect the difference between the multi-label segmentationmap output by CNN 400 and a corresponding ground truth multi-labelsegmentation map. The loss may be backpropogated through the layers ofCNN 400, starting from the output layer 456, and proceeding to inputlayer 402, wherein at each layer a gradient of the loss function isdetermined for each parameter of the layer, and each parameter isupdated based on the determined gradient.

It will be appreciated that the current disclosure encompasses neuralnetwork architectures comprising one or more regularization layers,including batch normalization layers, dropout layers, Gaussian noiselayers, and other regularization layers known in the art of machinelearning which may be used during training to mitigate overfitting andincrease training efficiency while reducing training duration.Regularization layers are used during training and deactivated orremoved during post training implementation of the CNN. These layers maybe interspersed between the layers/feature maps shown in FIG. 4, or mayreplace one or more of the shown layers/feature maps.

It should be understood that the architecture and configuration of CNN400 shown in FIG. 4 is for illustration, not for limitation. Anyappropriate neural network can be used herein for predicting anatomicalROI attribute maps from MR calibration images, such as ResNet, recurrentneural networks, General Regression Neural Network (GRNN), etc. One ormore specific embodiments of the present disclosure are described abovein order to provide a thorough understanding. These describedembodiments are only examples of systems and methods for determiningmulti-label segmentation maps from 3D CCTA images using a deep neuralnetwork. The skilled artisan will understand that specific detailsdescribed in the embodiments can be modified when being placed intopractice without deviating the spirit of the present disclosure.

Referring to FIG. 5, a flow chart of a method 500 for detecting lesionsin 3D CCTA images of coronary trees using deep neural networks is shown,according to an exemplary embodiment. Method 500 may be implemented bythe CCTA imaging system 100, imaging system 200, and/or CCTA imageprocessing system 302.

Method 500 begins at operation 502, where a 3D CCTA image is acquired bya CCTA imaging system. The 3D CCTA image may comprise a plurality ofvoxels, wherein each voxel includes an intensity value. The 3D CCA imagemay comprise a 3D CCTA image of a coronary tree.

At operation 504, the 3D CCTA image is input into an input layer of atrained deep neural network, which maps 3D CCTA image to a multi-labelsegmentation map. In some embodiments, the deep neural network is a CNN,such as CNN 400 illustrated schematically in FIG. 4. In someembodiments, each voxel intensity value of the 3D CCTA image is inputinto a distinct neuron of the input layer of the deep neural network.The input intensity values are propagated through the one or more hiddenlayers of the deep neural network, until reaching an output layer ofthe, wherein the neurons of the output layer output a plurality ofprobability scores, corresponding to a plurality of anatomical classes,for each voxel or sub-voxel of the input 3D CCTA image. The relationshipbetween two adjacent layers of the deep neural network, other than theinput layer, may be described as follows:

$Y_{j} = {f\left( {{\sum\limits_{i = 1}^{n}{W_{ji}X_{i}}} + B_{j}} \right)}$

Where X_(i) is the output of the i-th neuron of the preceding layer,Y_(j) is the j-th neuron of the subsequent layer, W_(ji) is the weight,and B_(j) is the bias. In some embodiments, the activation function ƒ isa rectified linear unit (ReLU) function, for example, plain ReLUfunction, leaky ReLU function, parametric ReLU function, etc.

In some embodiments, the output from the output layer of the deep neuralnetwork is of a same dimension as the input 3D CCTA image. In someembodiments, the dimension of the output from the output layer comprisesa larger dimension than the input 3D CCTA image, in other words, theoutput multi-label segmentation map may comprise a super-resolutionsegmentation map, having a greater spatial resolution than the input 3DCCTA image. The multi-label segmentation map may comprise a 3Dgrid/array of entries, wherein each entry comprises one probabilityscore for each of a set of pre-defined anatomical classes. Each entrymay correspond to a unique spatial region within the input 3D CCTAimage, and thereby each entry may designate a series of probabilityscores for each of a plurality of anatomical classes for each spatialregion within the 3D CCTA image.

At operation 506, a plurality of 1D parametric curves are generated foreach branch of the coronary tree imaged by the input 3D CCTA image. Insome embodiments, operation 506 comprises identifying each branch of thecoronary tree, determining a centerline for each branch of the coronarytree, and computing a plurality of 1D parametric curves for each pointalong the centerlines of each branch of the coronary tree. In oneexample, 1D parametric curves may show a plaque thickness, a lumenradius, a plaque density, a vessel wall thickness, a lipid corethickness, or other pre-defined parameters which may be derived from themulti-label segmentation map, as a function of length along a centerlineof a coronary branch, or as a function of angle around a point of acenterline of the coronary branch.

In some examples, the 1D parametric curves may show a parameter as afunction of length along the centerline of a coronary branch, and/or asa function of angle around the centerline of the coronary branch at agiven point along the centerline. In one example, the 1D parametriccurves indicate a lumen radius, plaque thickness, or otherpre-determined parameter, for each angle around a point along acenterline of a branch of the coronary tree. For a point along thecenterline of the coronary branch, an average, maximum, and minimum foreach pre-determined parameter, may be determined using the parametervalues for each angle determined at the point.

The parameters may be determined at pre-specified intervals along abranch of a coronary tree by taking a cross section of the multi-labelsegmentation map, through the branch, perpendicular to a direction ofextent of the branch, and determining the parameters at each anglearound the centerline. In one example, a cross section of a multi-labelsegmentation map may be taken at a point along a centerline of a branchof a coronary tree. The cross section may show the lumen as asubstantially circular opening, padded by vessel wall, and plaque. Aradius of the lumen may be determined at each angle around thecenterline, that is, as the lumen cross section may not be exactlycircular, and therefore different radii may be determined at differentangles around the centerline. Similarly, vessel wall, plaque, lipidcore, etc. may not be symmetrical about the centerline (as seen in crosssection through a coronary branch), and therefore these asymmetricalparameters may be assessed at each angle around the centerline. Anaverage of each parameter may be determined for each cross section of acoronary branch, by averaging the parameter values at each angle aroundthe centerline for the cross section. The minimum and maximum values foreach parameter may be similarly determined for each cross section of acoronary branch.

The 1D parametric curves may be generated based on the multi-labelsegmentation map produced in operation 506. For example, a lumen radiusat a point along a centerline (longitudinal axis) of a branch of acoronary tree may be determined using a multi-label segmentation map bydetermining a number of voxels classified as lumen along a line passingthrough, and perpendicular to, the centerline, and multiplying thenumber of voxels by a spatial size of each voxel.

At operation 508, the plurality of 1D parametric curves are used todetermine if one or more lesions are present along one or more of thebranches of the coronary tree. Further, at operation 508, in response toa detected lesion, the location of the lesion, including the start andstop points of the lesion along a branch of the coronary tree may bedetermined. In some embodiments, CCTA image processing system 302 maydetermine a lesion is present in a branch of a coronary tree responsiveto a lumen radius decreasing below a threshold lumen radius, and furtherresponsive to a plaque thickness exceeding a threshold plaque thickness,along section/length of a branch of the coronary tree. The location of adetected lesion, including the start and stop points of the lesion inthe branch of the coronary, tree may be determined based on the lumenradius and plaque thickness thresholds. In one example, a point along acenterline of a branch of a coronary tree may be set as the start pointof a lesion, in response to an average plaque thickness at the pointincreasing beyond a plaque thickness threshold, and further in responseto an average lumen radius at the point decreasing to below a lumenradius threshold. Similarly, a point along a centerline of a branch of acoronary tree may be set as the end point of a lesion in response to anaverage plaque thickness at the point decreasing to below a plaquethickness threshold, and further in response to an average lumen radiusat the point increasing to above a lumen radius threshold.

At operation 510, severity scores for the lesions detected at operation508 are determined. Severity score(s) for each of the detected lesion(s)may be determined based on one or more values of the plurality of 1Dparametric curves in the region(s) of the detected lesion(s). In someembodiments, a severity score for a detected lesion may be based on alumen radius and plaque thickness in a region of the lesion. In someembodiments, a severity score may be determined for a lesion based onthe plurality of 1D parametric curves by increasing a severity score asa lumen radius in a region of the lesion decreases below a lumen radiusthreshold. In some embodiments, a severity score is determined based ona thickness of plaque in a region of the lesion.

At operation 512, the detected lesions are displayed to a user via adisplay device, along with computed severity scores for each of thedetected lesions. In some embodiments, a detected lesion is shown to auser by overlaying an indication region on the input 3D CCTA image. Insome embodiments, a detected lesion is displayed to a user using theplurality of 1D parametric curves, wherein a region corresponding to theone or more detected lesions is highlighted, boxed, or otherwisedistinguished from other regions of the 1D parametric curves.

In this way, method 500 enables automatic, rapid, and robust detectionand designation of coronary lesions using deep neural networks. Atechnical effect of automatically labeling voxels of a 3D CCTA imageusing a trained deep neural network is that a time of diagnosis, and aconsistency of diagnosis of coronary lesions, may be increased. Further,a technical effect of automatically determining a severity score for theone or more detected coronary lesions based on 1D parametric curvesgenerated using the multi-label segmentation map, is that a moreconsistent, quantitative severity score scheme may be employed rapidly,without relying on manual assessment by expert cardiologists.

Referring to FIG. 6, a flow chart of a method 600 for training a deepneural network (such as CNN 400 shown in FIG. 4) is shown, according toan exemplary embodiment. Method 600 may be implemented by one or more ofthe above discussed systems. In some embodiments, method 600 may beimplemented by training module 310, stored in non-transitory memory 306of CCTA image processing system 302.

Method 600 begins at operation 602, where a training data pair, from aplurality of training data pairs, is fed to a deep neural network,wherein the training data pair comprises a 3D CCTA image of a coronarytree and a corresponding ground truth multi-label segmentation map ofthe coronary tree. In some embodiments, the training data pair, and theplurality of training data pairs, may be stored in CCTA image data 312.In other embodiments, the training data pair may be acquired viacommunicative coupling between the CCTA image processing system and anexternal storage device, such as via Internet connection to a remoteserver. In some embodiments, the ground truth multi-label segmentationmap is generated manually via expert curation.

At operation 604, the 3D CCTA image of the training data pair is inputinto an input layer of the deep neural network. In some embodiments, the3D CCTA image is input into an input layer of a CNN. In someembodiments, each voxel intensity value of the 3D CCTA image is inputinto a distinct neuron of the input layer of the deep neural network.

At operation 606, the deep neural network maps the input 3D CCTA imageto a 3D multi-label segmentation map by propagating the input 3D CCTAimage from the input layer, through one or more hidden layers, untilreaching an output layer of the deep neural network. In some embodimentsthe output of the deep neural network comprises a 3D matrix of entries,wherein each entry corresponds to a distinct voxel of the input 3D CCTAimage, and wherein each entry comprises a plurality of probabilityscores/label scores for each of a plurality of pre-determined anatomicalclasses.

At operation 608, the difference between the predicted multi-labelsegmentation map and the ground truth multi-label segmentation map ofthe training data pair is calculated.

At operation 610, the weights and biases of the deep neural network areadjusted based on the difference between the predicted multi-labelsegmentation map and the ground truth multi-label segmentation map. Thedifference (or loss), as determined by the loss function, may be backpropagated through the neural learning network to update the weights(and biases) of the convolutional layers. In some embodiments, backpropagation of the loss may occur according to a gradient descentalgorithm, wherein a gradient of the loss function (a first derivative,or approximation of the first derivative) is determined for each weightand bias of the deep neural network. Each weight (and bias) of the deepneural network is then updated by adding the negative of the product ofthe gradient determined (or approximated) for the weight (or bias) witha predetermined step size. Method 600 may then end. It will be notedthat method 600 may be repeated until the weights and biases of the deepneural network converge, or the rate of change of the weights and/orbiases of the deep neural network for each iteration of method 500 areunder a threshold.

In this way, method 600 enables a deep neural network to be trained topredict a multi-label segmentation map from an input 3D CCTA image of acoronary tree.

Turning to FIG. 700, an example of a plurality of 1D parametric curves700 generated for a branch of a coronary tree using a multi-labelsegmentation map of the coronary tree, is shown. The plurality of 1Dparametric curves 700 may be generated by one or more of the systemsdescribed above, using a multi-label segmentation map generated by adeep neural network using one or more 3D CCTA images. In one embodiment,parametric curve module 314, stored in non-transitory memory 306 of CCTAimage processing system 302, may produce the plurality of 1D parametriccurves 700 from a multi-label segmentation map produced by a deep neuralnetwork, such as deep neural network 400, illustrated in FIG. 4.

1D parametric curves 700 include a plaque thickness 702, and a lumenradius 706, determined at points along a centerline of one branch of acoronary tree. In other words, 1D parametric curves 700 comprise plaquethickness 702 and lumen radius 706, determined as a function of distancealong a branch of a coronary tree. In one example, plaque thickness 702may comprise an average plaque thickness, determined at each point alonga centerline of a branch of the coronary tree, determined as an averageof the plaque thickness at each angle around each point of thecenterline. Similarly, lumen radius 706 may comprise an average lumenradius, determined at each point along the centerline, as the average ofeach lumen radius determined for each angle around the centerline.

Although 1D parametric curves 700 include plaque thickness 702, andlumen radius 706, for a single coronary branch of a coronary tree, itwill be appreciated that the current disclosure provides for 1Dparametric curves determined for more than one branch of a coronary tree(e.g., a plurality of 1D parametric curves similar to those shown inFIG. 7 may be determined for each of a plurality of branches of acoronary tree), and wherein the 1D parametric curves may comprisesubstantially any pre-defined parameter herein disclosed, which may bederived from the multi-label segmentation maps as a function of distancealong a centerline of a branch of a coronary tree, including radius,diameter, thickness, area, and volume of any of the labeled components(lumen, vessel wall, plaque, lipid core, fibrous cap, etc.) included inthe 3D multi-label segmentation map, determined at a point, or at arunning average of points, along a centerline of a branch of a coronarytree. In some embodiments, 1D parametric curves determined for one ormore branches of a coronary tree may include, lumen volume, lipid corethickness, plaque volume, plaque area, lumen tortuosity, vessel wallthickness, and plaque roughness/shape.

1D parametric curves 700 include plaque thickness 702, showing theplaque thickness at each point along a centerline of the branch of thecoronary tree. In some embodiments, plaque thickness 702 may bedetermined using a 3D multi-label segmentation map, such as thosedescribe throughout the current disclosure, according to apre-determined/pre-defined algorithm, and therefore, plaque thicknessmay be referred to as pre-determined parameter. In one embodiment,determining a plaque thickness at a point along a centerline of a branchof a coronary tree included in a 3D multi-label segmentation mapcomprises, fitting a centerline through each distinct path/branch of acoronary tree, by minimizing a sum of squared distances between thecenterline and each voxel of the multi-label segmentation map classifiedas vessel wall within a plane perpendicular to a longitudinal extent ofthe coronary branch, and at each point along each centerline of each ofthe branches, determining a cross sectional thickness of plaque, whereinthe cross section is perpendicular to a direction of extent of thecenterline. In some embodiments, determining a cross sectional thicknessof plaque at a point along a centerline of a branch of a coronary treecomprises determining a number of voxels with a plane perpendicular to adirection of extent of the centerline, and multiplying the number ofvoxels by a constant, wherein the constant is the ratio between lengthand voxel (e.g., 1 mm/voxel).

Plaque thickness 702 is shown along with plaque thickness threshold 704,wherein plaque thickness threshold 704 comprises a pre-determined plaquethickness threshold. Plaque thickness threshold 704 may be determinedbased on one or more parameters of the coronary branch for which 1Dparametric curves are determined. In one embodiment, plaque thicknessthreshold 704 may comprise a pre-determined fraction of a current lumenradius of the current branch, or a pre-determined fraction of an averagelumen radius for the current branch. In some embodiments, plaquethickness threshold 704 may comprise a constant value. At L₁, plaquethickness 702 increases beyond plaque thickness threshold 704, which insome embodiments, may indicate a starting point of a coronary lesionwithin the current coronary branch. At L₂, plaque thickness 702decreases to below plaque thickness threshold 704, which in someembodiments, may indicate a stopping/termination point of the coronarylesion. In some examples, a location of a lesion may be specified by thestart and stop point of the lesion, such as may be indicated by L₁ andL₂. In some embodiments, a severity score for the coronary lesionindicated between L₁ and L₂ may be determined based on the extent beyondthe plaque thickness threshold 704 to which plaque thickness 702extends, wherein the severity score may increase as the plaque thickness702 increases beyond plaque thickness threshold 704.

1D parametric curves 700 further include lumen radius 706, showing thelumen radius at each point along the centerline of the branch of thecoronary tree. In some embodiments, lumen radius 706 may be determinedusing a 3D multi-label segmentation map, according to pre-determinedinstructions, and therefore, lumen radius may be referred to aspre-determined parameter. In one embodiment, determining a lumen radiusat a point along the centerline of a branch of a coronary tree includedin a 3D multi-label segmentation map comprises, fitting a centerlinethrough each distinct path/branch of a coronary tree, by minimizing asum of squared distances between the centerline and each voxel of themulti-label segmentation map classified as vessel wall within a planeperpendicular to a longitudinal extent of the coronary branch, and ateach point along each centerline of each of the branches of the coronarytree, determining a cross sectional diameter/radius of the lumen,wherein the cross section is perpendicular to a direction of extent ofthe centerline. In some embodiments, determining a lumen radius at apoint along a centerline of a branch of a coronary tree using amulti-label segmentation map comprises, determining a number of voxelsin the multi-label segmentation map classified as lumen within a planeperpendicular to a direction of extent of the centerline, andmultiplying the number of voxels by a constant, wherein the constant isthe ratio between length and voxel (e.g., 1 mm/voxel).

Shown alongside lumen radius 706 is lumen radius threshold 708. Lumenradius threshold 708 may be determined based on one or more parametersof the coronary branch for which 1D parametric curves 700 aredetermined. In one embodiment, lumen radius threshold 708 may comprise apre-determined fraction of a current lumen radius of the current branch,or a pre-determined fraction of an average lumen radius for the currentbranch. In some embodiments, lumen radius threshold 708 may comprise aconstant, pre-determined value. At L₁, lumen radius 706 decreases tobelow lumen radius threshold 708, which in some embodiments, mayindicate a starting point of a coronary lesion within the currentcoronary branch. At L₂, lumen radius 706 increases above lumen radiusthreshold 708, which in some embodiments, may indicate astopping/termination point of the coronary lesion. In some embodiments,a severity score for the coronary lesion indicated between L₁ and L₂ maybe determined based on the extent beyond the lumen radius threshold 708to which lumen radius 706 extends, wherein the severity score mayincrease as the lumen radius 706 decreases below the lumen radiusthreshold 708.

In some embodiments, a coronary lesion may detected responsive to bothplaque thickness 702 increasing above plaque thickness threshold 704,and lumen radius 706 decreasing to below the lumen radius threshold 708,in a same region along a branch of the coronary tree, such as between L₁and L₂, in FIG. 7.

Turning to FIG. 8, three examples of CCTA images, and correspondingmulti-label segmentation maps, are shown. Specifically, FIG. 8 showsfirst CCTA image 802, second CCTA image 806, and third CCTA image 810,which may be mapped via a 3D convolutional neural network (e.g., CNN400), to first multi-label segmentation map 804, second multi-labelsegmentation map 808, and third multi-label segmentation map 812,respectively. Each of the multi-label segmentation maps (804, 808, and812), include labeled regions of plaque (pink), lumen (greenish-blue),lipid core (yellow), and external tissue (blue), which are clearlyindicated by the associated, colored labels. The CCTA images of FIG. 8may be acquired by an imaging system, such as CCTA imaging system 100,shown in FIG. 1. Further, the multi-label segmentation maps (804, 808,and 812) may be used according to one or more methods disclosed herein,to generate 1D parametric curves, from which a location and severity ofcoronary lesions may be ascertained.

First CCTA image 802 shows a first branch of a coronary tree. First CCTAimage 802 may be mapped to first multi-label segmentation map 804 via atrained convolutional neural network. First multi-label segmentation map804 includes a coronary lesion start point 816, and a coronary lesionend point 818, with plaque 814 and lumen 820 therebetween. Plaque 814comprises voxels of first CCTA image 802 classified as plaque (shown inpink), while lumen 820 comprises voxels of first CCTA image 802classified as lumen (shown in greenish-blue). The region of the coronarybranch bounded by start point 816 and end point 818 comprises anidentified coronary lesion, having a reduced lumen radius therein,indicating a region of plaque buildup along the branch of the coronarytree.

Second CCTA image 806 shows a second branch of a coronary tree includinga lesion. Second CCTA image 806 may be mapped to second multi-labelsegmentation map 808 via a trained convolutional neural network. Secondmulti-label segmentation map 808 includes a coronary lesion start point822, and a coronary lesion end point 830, with lumen 824, plaque 826,and external tissue 828, shown therebetween. Lumen 824 comprises voxelsof second CCTA image 806 classified/labeled as lumen (greenish-blue),plaque 826 comprises voxels of second CCTA image 806 classified asplaque (shown in pink), while external tissue 828 comprises voxels ofsecond CCTA image 806 classified as external tissue (blue). The regionof the second coronary branch bounded by start point 822 and end point830 comprises an identified coronary lesion, having a reduced lumenradius therein, indicating a region of plaque buildup along the branchof the coronary tree.

Third CCTA image 810 shows a third branch of a coronary tree including alesion. Third CCTA image 810 may be mapped to third multi-labelsegmentation map 812 via a trained convolutional neural network. Thirdmulti-label segmentation map 812 includes a coronary lesion start point832, and a coronary lesion end point 840, with lumen 834, lipid core836, and plaque 838, shown therebetween. Lumen 834 comprises voxels ofthird CCTA image 810 classified/labeled as lumen (greenish-blue), lipidcore 836 comprises voxels of third CCTA image 810 classified/labeled aslipid core (yellow), and plaque 838 comprises voxels of third CCTA image810 classified/labeled as plaque (shown in pink). The region of thethird coronary branch bounded by start point 832 and end point 840comprises an identified coronary lesion, having a reduced lumen radiustherein, indicating a region of plaque buildup along the branch of thecoronary tree.

When introducing elements of various embodiments of the presentdisclosure, the articles “a,” “an,” and “the” are intended to mean thatthere are one or more of the elements. The terms “first,” “second,” andthe like, do not denote any order, quantity, or importance, but ratherare used to distinguish one element from another. The terms“comprising,” “including,” and “having” are intended to be inclusive andmean that there may be additional elements other than the listedelements. As the terms “connected to,” “coupled to,” etc. are usedherein, one object (e.g., a material, element, structure, member, etc.)can be connected to or coupled to another object regardless of whetherthe one object is directly connected or coupled to the other object orwhether there are one or more intervening objects between the one objectand the other object. In addition, it should be understood thatreferences to “one embodiment” or “an embodiment” of the presentdisclosure are not intended to be interpreted as excluding the existenceof additional embodiments that also incorporate the recited features.

In addition to any previously indicated modification, numerous othervariations and alternative arrangements may be devised by those skilledin the art without departing from the spirit and scope of thisdescription, and appended claims are intended to cover suchmodifications and arrangements. Thus, while the information has beendescribed above with particularity and detail in connection with what ispresently deemed to be the most practical and preferred aspects, it willbe apparent to those of ordinary skill in the art that numerousmodifications, including, but not limited to, form, function, manner ofoperation and use may be made without departing from the principles andconcepts set forth herein. Also, as used herein, the examples andembodiments, in all respects, are meant to be illustrative only andshould not be construed to be limiting in any manner.

1. A method comprising: acquiring a 3D cardiac computed tomography andangiography (CCTA) image of a coronary tree; mapping the 3D CCTA imageto a multi-label segmentation map with a trained deep neural network;generating a plurality of 1D parametric curves for a branch of thecoronary tree using the multi-label segmentation map; determining alocation of a lesion in the branch of the coronary tree using theplurality of 1D parametric curves; and determining a severity score forthe lesion based on the plurality of 1D parametric curves, whereingenerating the plurality of 1D parametric curves for the branch of thecoronary tree using the multi-label segmentation map comprisesdetermining values for each of a plurality of a pre-defined parametersfor each point along a centerline of the branch of the coronary tree,the plurality of pre-defined parameters include one or more of lipidcore thickness, plaque thickness, plaque area, plaque density, vesselwall thickness, lumen radius, lumen tortuosity, and lumen volume.
 2. Themethod of claim 1, wherein the multi-label segmentation map comprises aplurality of probability scores for each voxel of the 3D CCTA image, fora plurality of anatomical classes.
 3. The method of claim 2, wherein thepre-defined parameters are determined using a cross section of themulti-label segmentation map at each point along the centerline of thebranch of the coronary.
 4. The method of claim 1, wherein determiningthe location of the lesion in the branch of the coronary tree using theplurality of 1D parametric curves comprises: determining start and stoppoints of the lesion in the branch of the coronary tree based on theplurality of 1D parametric curves.
 5. The method of claim 1, wherein ateach point along the centerline, minimum, maximum, and average valuesfor one or more of lipid core thickness, plaque thickness, plaque area,plaque density, vessel wall thickness, lumen radius, lumen tortuosity,and lumen volume, are determined for each angle around the centerline.6. The method of claim 2, wherein the plurality of anatomical classescomprise: lumen; media; lipid core; fibrous cap; calcium; and externaltissue.
 7. The method of claim 1, wherein determining the location ofthe lesion in the branch of the coronary tree using the plurality of 1Dparametric curves comprises: responding to a plaque thickness beinggreater than a plaque thickness threshold in a region of the branch by:concluding the region is the location of the coronary lesion.
 8. Themethod of claim 7, wherein determining the severity score for the lesionbased on the plurality of 1D parametric curves comprises, determiningthe severity score based on a lipid core thickness in the region.
 9. Themethod of claim 1, wherein a resolution of the multi-label segmentationmap is greater than a native resolution of the 3D CCTA image.
 10. Themethod of claim 1, wherein the trained deep neural network comprises a3D convolutional neural network.
 11. A method comprising: training adeep neural network to map 3D CCTA images to 3D multi-label segmentationmaps; receiving a first 3D CCTA image; mapping the first 3D CCTA imageto a 3D multi-label segmentation map using the trained deep neuralnetwork; and determining a plurality of 1D parametric curves for acoronary artery included in the 3D CCTA image using the 3D multi-labelsegmentation map, wherein determining the plurality of 1D parametriccurves for the coronary artery comprises determining values for each ofa plurality of a pre-defined parameters for each point along acenterline of the branch of the coronary tree, the plurality ofpre-defined parameters include lipid core thickness, plaque thickness,plaque area, plaque density, vessel wall thickness, lumen radius, lumentortuosity, and lumen volume.
 12. The method of claim 11, whereintraining the deep neural network comprises feeding a training data pairto the deep neural network, wherein the training data pair includes asecond 3D CCTA image and a corresponding ground truth 3D multi-labelsegmentation map.
 13. The method of claim 12, wherein training the deepneural network comprises: mapping the second 3D CCTA image in thetraining data pair to a predicted 3D multi-label segmentation map usingthe deep neural network; calculating a difference between the predicted3D multi-label segmentation map and the ground truth 3D multi-labelsegmentation map; and adjusting parameters of the deep neural networkvia backpropagation based on the difference between the predicted 3Dmulti-label segmentation map and the ground truth 3D multi-labelsegmentation map.
 14. A cardiac computed tomography and angiography(CCTA) imaging system comprising: an x-ray radiation source; an x-raydetector array; a memory storing a trained deep neural network andinstructions; and a processor communicably coupled to the x-rayradiation source, the x-ray detector array, and the memory, and whenexecuting the instructions, configured to: acquire a 3D CCTA image of acoronary tree using the x-ray radiation source and the x-ray detectorarray; map the 3D CCTA image to a multi-label segmentation map with thetrained deep neural network; generate a plurality of 1D parametriccurves for a branch of the coronary tree using the multi-labelsegmentation map; and determine a location of a lesion in the branch ofthe coronary tree using the plurality of 1D parametric curves, whereinthe processor is configured to generate the plurality of 1D parametriccurves for the branch of the coronary tree using the multi-labelsegmentation map by determining values for each of a plurality of apre-defined parameters along a centerline of the branch of the coronarytree, wherein the pre-defined parameters are derived from themulti-label segmentation map, wherein the plurality of pre-definedparameters include one or more of lipid core thickness, plaquethickness, plaque area, plaque density, vessel wall thickness, lumenradius, lumen tortuosity, and lumen volume.
 15. The system of claim 14,wherein the multi-label segmentation map comprises a plurality ofprobability scores for each voxel of the 3D CCTA image, for each of aplurality of anatomical classes, wherein the plurality of anatomicalclasses include one or more of lumen, media, lipid core, fibrous cap,calcium, and external tissue.
 16. The system of claim 14, wherein, whenexecuting the instructions, the processor is further configured to:determine a severity score for the lesion based on a lipid corethickness.
 17. The system of claim 16, wherein the system furthercomprises a display device, and wherein, when executing theinstructions, the processor is further configured to display thelocation of the lesion and the severity score via the display device.18. The system of claim 14, wherein the processor is configured todetermine the location of the lesion in the branch of the coronary treeusing the plurality of 1D parametric curves by: comparing the pluralityof 1D parametric curves against one or more pre-defined thresholds; anddetermine start and stop points of the lesion in the branch of thecoronary tree based on one or more of the plurality of 1D parametriccurves crossing one or more of the pre-defined thresholds.
 19. Thesystem of claim 14, wherein the processor is configured to determine thelocation of the lesion in the branch of the coronary tree using theplurality of 1D parametric curves by: responding to both a plaquethickness being greater than a plaque thickness threshold and a lumenradius being less than a lumen radius threshold, in a region of a branchof the coronary tree by: flagging the region as including a lesion. 20.The system of claim 19, wherein the lumen radius threshold is determinedbased on an average lumen radius for the branch of the coronary tree.